Integration of machine learning and first principles models

Model building and parameter estimation are traditional concepts widely used in chemical, biological, metallurgical, and manufacturing industries. Early modeling methodologies focused on mathematically capturing the process knowledge and domain expertise of the modeler. The models thus developed are termed first principles models (or white-box models). Over time, computational power became cheaper, and massive amounts of data became available for modeling. This led to the development of cutting edge machine learning models (black-box models) and artificial intelligence (AI) techniques. Hybrid models (gray-box models) are a combination of first principles and machine learning models. The development of hybrid models has captured the attention of researchers as this combines the best of both modeling paradigms. Recent attention to this field stems from the interest in explainable AI (XAI), a critical requirement as AI systems become more pervasive. This work aims at identifying and categorizing various hybrid models available in the literature that integrate machine-learning models with different forms of domain knowledge. Benefits such as enhanced predictive power, extrapolation capabilities, and other advantages of combining the two approaches are summarized. The goal of this article is to consolidate the published corpus in the area of hybrid modeling and develop a comprehensive framework to understand the various techniques presented. This framework can further be used as the foundation to explore rational associations between several models.     

Homotopy continuation enhanced branch and bound algorithms for strongly nonconvex mixed-integer nonlinear optimization

Large-scale strongly nonlinear and nonconvex mixed-integer nonlinear programming (MINLP) models frequently appear in optimization-based process synthesis, integration, intensification, and process control. However, they are usually difficult to solve by existing algorithms within acceptable time. In this study, we propose two robust homotopy continuation enhanced branch and bound (HCBB) algorithms (denoted as HCBB-FP and HCBB-RB) where the homotopy continuation method is employed to gradually approach the optimum of the NLP subproblem at a node from the solution at its parent node. A variable step length is adapted to effectively balance feasibility and computational efficiency. The computational results from solving four existing process synthesis problems demonstrate that the proposed HCBB algorithms can find the same optimal solution from different initial points, while the existing MINLP algorithms fail or find much worse solutions. In addition, HCBB-RB is superior to HCBB-FP due to much lower computational effort required for the same locally optimal solution.     

Development of silicon wafer packaging technology for deep UV LED

This paper reports a deep‐ultraviolet LED (deep‐UV‐LED) package based on silicon MEMS process technology (Si‐PKG). The package consists of a cavity formed by silicon crystalline anisotropic etching, through‐silicon vias (TSVs) filled with electroplated Cu, bonding metals made of electroplated Ni/AuSn and a quartz lid for hermetic sealing. A deep‐UV LED die is directly mounted in the Si‐PKG by AuSn eutectic bonding without a submount. It has advantages in terms of size, heat dissipation, light utilization efficiency, productivity and cost over conventional AlN ceramic packages. We confirmed a light output of 30 mW and effective reflection on Si (111) cavity slopes in the Si‐PKG. Based on simulation, further improvement of the optical output is expected by optimizing DUV‐LED die mount condition.     

Input-to-state learning of recurrent neural networks with delay and disturbance

This article deals with the issue of input-to-state stabilization for recurrent neural networks with delay and external disturbance. The goal is to design a suitable weight-learning law to make the considered network input-to-state stable with a predefined -gain. Based on the solution of linear matrix inequalities, two schemes for the desired learning law are presented via using decay-rate-dependent and decay-rate-independent Lyapunov functionals, respectively. It is shown that, in the absence of external disturbance, the proposed learning law also guarantees the exponential stability of the network. To illustrate the applicability of the present weight-learning law, two numerical examples with simulations are given.     

Machine Learning-Based Two-Stage Data Selection Scheme for Long-Term Influenza Forecasting

One popular strategy to reduce the enormous number of illnesses and deaths from a seasonal influenza pandemic is to obtain the influenza vaccine on time. Usually, vaccine production preparation must be done at least six months in advance, and accurate long-term influenza forecasting is essential for this. Although diverse machine learning models have been proposed for influenza forecasting, they focus on short-term forecasting, and their performance is too dependent on input variables. For a country’s long-term influenza forecasting, typical surveillance data are known to be more effective than diverse external data on the Internet. We propose a two-stage data selection scheme for worldwide surveillance data to construct a long-term forecasting model for influenza in the target country. In the first stage, using a simple forecasting model based on the country’s surveillance data, we measured the change in performance by adding surveillance data from other countries, shifted by up to 52 weeks. In the second stage, for each set of surveillance data sorted by accuracy, we incrementally added data as input if the data have a positive effect on the performance of the forecasting model in the first stage. Using the selected surveillance data, we trained a new long-term forecasting model for influenza and perform influenza forecasting for the target country. We conducted extensive experiments using six machine learning models for the three target countries to verify the effectiveness of the proposed method. We report some of the results.     

Image-Based Lifelogging: User Emotion Perspective

Lifelog is a digital record of an individual’s daily life. It collects, records, and archives a large amount of unstructured data; therefore, techniques are required to organize and summarize those data for easy retrieval. Lifelogging has been utilized for diverse applications including healthcare, self-tracking, and entertainment, among others. With regard to the image-based lifelogging, even though most users prefer to present photos with facial expressions that allow us to infer their emotions, there have been few studies on lifelogging techniques that focus upon users’ emotions. In this paper, we develop a system that extracts users’ own photos from their smartphones and configures their lifelogs with a focus on their emotions. We design an emotion classifier based on convolutional neural networks (CNN) to predict the users’ emotions. To train the model, we create a new dataset by collecting facial images from the CelebFaces Attributes (CelebA) dataset and labeling their facial emotion expressions, and by integrating parts of the Radboud Faces Database (RaFD). Our dataset consists of 4,715 high-resolution images. We propose Representative Emotional Data Extraction Scheme (REDES) to select representative photos based on inferring users’ emotions from their facial expressions. In addition, we develop a system that allows users to easily configure diaries for a special day and summaize their lifelogs. Our experimental results show that our method is able to effectively incorporate emotions into lifelog, allowing an enriched experience.     

Prediction of rockhead using a hybrid N-XGBoost machine learning framework

The spatial information of rockhead is crucial for the design and construction of tunneling or underground excavation. Although the conventional site investigation methods (i.e. borehole drilling) could provide local engineering geological information, the accurate prediction of the rockhead position with limited borehole data is still challenging due to its spatial variation and great uncertainties involved. With the development of computer science, machine learning (ML) has been proved to be a promising way to avoid subjective judgments by human beings and to establish complex relationships with mega data automatically. However, few studies have been reported on the adoption of ML models for the prediction of the rockhead position. In this paper, we proposed a robust probabilistic ML model for predicting the rockhead distribution using the spatial geographic information. The framework of the natural gradient boosting (NGBoost) algorithm combined with the extreme gradient boosting (XGBoost) is used as the basic learner. The XGBoost model was also compared with some other ML models such as the gradient boosting regression tree (GBRT), the light gradient boosting machine (LightGBM), the multivariate linear regression (MLR), the artificial neural network (ANN), and the support vector machine (SVM). The results demonstrate that the XGBoost algorithm, the core algorithm of the probabilistic N-XGBoost model, outperformed the other conventional ML models with a coefficient of determination (R²) of 0.89 and a root mean squared error (RMSE) of 5.8 m for the prediction of rockhead position based on limited borehole data. The probabilistic N-XGBoost model not only achieved a higher prediction accuracy, but also provided a predictive estimation of the uncertainty. Thus, the proposed N-XGBoost probabilistic model has the potential to be used as a reliable and effective ML algorithm for the prediction of rockhead position in rock and geotechnical engineering.